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The present invention relates to a method and an apparatus for 
producing a high-resolution image. 

Solid state imaging devices such as CCDs (Charge Coupled 
5 Devices) or C-MOS arrays consisting of imaging surfaces, with 
pixels arranged in a one or two-dimensional matrix are widely- 
used in digital cameras and scanners. 

There is however a growing demand for higher resolution of the 
10 images. One way of obtaining a higher resolution is by 
increasing the number of pixels in the imaging surface of solid 
state imaging devices. However, if the number of pixels is 
simply increased in e.g. a CCD to satisfy the need for high 
resolution, its chip size has to be further increased. On the 
15 other hand, in order to increase the number of pixels without 
changing the present CCD chip size, the integration of pixels 
must be greatly enhanced. It is technically difficult to 
prepare such a high-density CCD. Even if development of 
improved fabrication techniques allows the manufacture of such 
20 a CCD in the near future, the circuit configuration for driving 
such a CCD becomes complicated and power consumption becomes 
higher, thus entailing new problems, which will result in loss 
of production efficiency and higher cost. 

25 Other methods for obtaining higher resolution have been used. 
In scanners the CCD or CMOS array are usually moved in one 
direction in steps equal to the height of the individual 
sensors by means of stepper motors , and in special high 
resolution cameras, the sensor matrix are moved in both an X 

30 and an Y directions, to move the individual cells to positions 
not covered by any cell in the matrix (due to a physical 
distance between each sensor) or to positions not covered by 
either a red, green og blue filtered cell. All methods have the 
limitation that the area of each sensor-cell is limiting the 

35 achievable resolution. 
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US 6,002,810 disclose a method for generating an image having 
an arbitrary number of pixels starting from a given digitally 
image having a given number of pixels. This method does not 
provide a means for iinproving the resolution and the quality of 
5 the image after enlarging will be inferior to the original. 

US 4,652,928 relate to a method for improving the actual 
resolution of an image at the time the image is being 
established. However, the improvement of the resolution is 
10 limited by a factor 2 and is effected solely upon movement in a 
horizontal direction. Moreover, as already mentioned the 
achievable resolution is limited by the size of the sensor- 
cells. 

15 It is therefore an object of the present invention to provide a 
new and improved method and apparatus capable of attaining 
higher resolution and realising better image quality cheaper 
than known up till now. 

20 According to the invention a low resolution first image is 
converted to a high resolution second image by means of a light 
sensor for receiving incident light radiated from a scene, 
whereby said light sensor is constructed as an array or a 
matrix having a number of cells, each defining a predetermined 

25 area. 

The method for carrying out said converting is comprising that 
said first image is cyclically scanned a number of times, chat 
the light sensor simultaneously with the scanning stepwise is 

30 moved an identical number of times relative to the light 
radiating scene in at least one direction thereby defining a 
number of subareas, that the total distance covered during 
movement of the light sensor corresponds to the extend of the 
cell or to to the extend of the cell plus a distance defined by 

35 or to the neighbour cell in said at least one direction , and 
that a representation of the high resolution second image is 
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established toy calculating the representation of the received 
incident light from the scene at each subarea. 

Thereby a method advantageously is obtained in which the 
5 calculations are limited to a minimum but were the image 
resolution is only limited by factors as optics and precision 
of the positioning mechanism and not by the number and size of 
each sensor cell, thus giving the possibility of achieving an 
extremely high resolution, not known by any other method. 

10 

The calculating representing the received incident light from 
the scene at each subarea can advantageously be performed by 
means of a coinputer having a software program for receiving the 
output from the position sensor and on basis of this output 
15 calculate control values for bringing the light sensor to move 
in a wanted way. 

If the light sensor for each step is moved a distance 
corresponding to the extend of the area covered by the cell in 
20 the direction of movement, divided with the number of scanning 
times it is possible, due to the overlapping between the 
scannings, to have several scannings on the same cell whereby 
the calculations in the cycle and thus the resolution will 
become even more accurate. 

25 

By an advantageously embodiment according to the invention the 
light sensor can be stepwise moved by means of at least one 
driver relative to the light radiating scene, the immediate 
position of the light sensor can be recorded by means of at 

30 least one position sensor generating output representing said 
sensor position, said output can be send to a computer having a 
software program for calculating control values on basis of the 
received output, and signals representing the calculated 
control values can be send to the at least one position driver 

35 for bringing this to drive the light sensor in such a way that 
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the steps of movement as function of the scannings defines a 
predetermined curve in a coordinate system. 

The predetermined curve can have any appropriate form such as a 
5 stairway or be sinusoidal. 

Another preferred form is a triangle. By choosing a triangle 
curve the scannings can be performed with equidistant timely 
distances while the distance between the positions of the light 
10 sensor is identical. 

In practise the light sensor can be movable mounted on a frame 
and cyclically moved along this by means of the drivers 
simultaneously with scanning the low resolution first image a 
15 number of times by means of an activator bringing the light 
sensor to scan. 

The values calculated after all predetermined number of 
scannings can advantageously be computed by means of the 
20 following formula? 

i.(z) ■ V.+A/aln) - SI 4 (n> for n=l to a=N, n*z (1) 
where 

25 

N is the number of subareas that each cell is splitted into, 
n is the index for subareas in the sensor cell, 
a is the area of the n'th subarea of the sensor cell, 
V ^ is the output signal from the sensor cells, 
30 I % is the computed light radiation received by the subarea a, 
z is the subarea to be calculated, 
A is the total area of the sensor cell. 

This formula is based on the assumption that the output from 
35 each imaging surface of the individual sensor cells arranged in 
a two-dimensional matrix is in principle a representation of 
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the integration of the light intensity on the area of the 
optical sensor elements: 

V„ = Ji,<a)da. 

5 

When there is a definite number of subareas a in A, is the 
formula : 

v e = XI. <n) for n=l to N, 
where 

N is the number of subareas that each cell is splitted into, 
n is the index for subareas in the sensor cell, 
15 V Q is the output signals from the sensor cells, 

I, is the computed light radiation received by the subarea a. 

The larger the sensor element is the larger will V„ be. It is 
however necessary that V 0 is independent of the area of the 
20 sensor, since a given light intensity always must represent the 
same V D . Therefore, V # is scaled in accordance with the pixel 
area so: 

V p = Sl # (n) for n=l to N, *> 
25 V, = (l/A)* £ljn)*a(n> for n=l to N, 

where a(n) is the area of the n'th subarea. 

This will of course happen automatically in the sensor via 
exposure time and related technique in the sensor, but the 
30 assumption is important for arriving to formula (1) . 

I # for a given subarea z is therefore: 

z «<z) ■ V Q *A/aU) - Xl # <n) for n«l to n=N, n*z (2) 

35 
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where 

N is the number of subareas that each cell is splitted into, . 
n is the index for subareas in the sensor cell. 
5 a is the area of the n'th subarea of the sensor cell. 
V D is the output signals from the sensor cells, 
I. is the computed light radiation received by the subarea a, 
z is the subarea to be used for calculation, 
A is the area of the sensor cell. 

10 

If all subareas have equal area, then A/a(z)»N and (1) can be 
written; 

l.(z) a V 0 *N - 2l.{n) for nol to n=N, a*z (3) 
15 The formula in (1> can in general work for any subarea in the 
entire sensor matrix, meaning that the summation of subareas 
can cover subareas from other sensor cells: 

I.(z) = V 0 *A/a(z) - £x.(n) for n*l to n=N, n*z (4) 

20 

where 

N is the number of subareas that the summation includes, 
n is the index for subareas to be summarised, 
25 a is the area of the n'th subarea, 

V* is the output signal from one of the sensor cells, which 
includes the subarea a(z), 

I, is the computed light radiation received by the subarea a, 
z is the subarea to be used for calculation, 
30 A is the area of the entire subareas. 

Since the method according to the invention is based on 
calculated values, or in some instances on estimated values, 
there will inevitably be generated errors in the calculations. 

35 
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In a preferred embodiment does the method therefore comprise a 
digital filter for minimising the errors. The digital filter 
can e.g. use several values from several different set of input 
values and the set of previous calculated output values to 
5 compute a new output value. This will minimise the influence 
from errors in calculated values or errors in an output value 
from a sensor cell. Different types of filters can be used, but 
the most simple type is based on the formula in (2) , where each 
factor on the right side is weighted with a predetermined 
10 value. 

In one embodiment of the invention is it assumed that the light 
sensor is formed as a x,Y-matrix, with respectively X*Y cells. 

15 When the sensor has been moved to a N number of new positions 
and a N number of sets of output values from the sensor has 
been stored, a new x,,Y,-matrix with X,*Y r cells can be 
calculated. The new matrix having a row of X*S cells and a 
column of T*Y cells, giving N<=S*T, and the digital filter as a 

20 modification of formula (3) can be written in the form: 

I. <x # . y,)= F v * V 0 Ix,y> - 2l # (x, - n . y,-m>*F(x # -n, y t -m) (4) 

for n=l to n= x, and m-1 to m-y, ; (n,m) * (x # .y e ) 

where 

25 

I. <x # ,y m ) is the light radiation value of the x>,y t element in 
the new X, . Y T matrix to be calculated, 

v e <x,y) is the sensor output value from the sensor element in 
the chosen scanning position covering the (x.,y t > subarea, 
30 F„ is the predetermined filter coefficient related to the 
sensor output value from the sensor element in the chosen 
scanning position covering the (x^,y t ) subarea, 

I.(x, -n, y t -m) is previous calculated I. values, in the new x.,y e 
matrix, 



10 



11/09/07 16:47 HOLfC PATENT ft/S NR. 915 Oil 



P(x D -n, y,-m) is the predetermined filter coefficients related 
to the previous calculated I. values for subareas in the new 
x B ,y^ matrix, 

n,m is the number of previous calculated l m values for subareas 
5 in the new x # ,y fi matrix, in the X and Y direction respectively. 

This digital filter has the advantage that the filter 
coefficients for previous calculated values of I. can be 
smaller and smaller, thereby ensuring that the influence of 
10 errors in previous calculated values are minimized. 

The method can by another advantageously embodiment according 
to the invention also be carried out in such a way that the 
scanning and the stepwise movement of the light sensor are 
15 asynchronous in relation to each other in the sense that each 
new scanning is taken place in a sensor position not scanned 
before whereby an almost infinite high resolution is possible 
by only scanning a sufficient number of times. 

20 The invention also relates to an apparatus for performing the 
method according to the invention. 

This apparatus correspond in general to the known solid state 
imaging devices known by the skilled person but with the 
25 exception that the optical sensor is arranged in a flexible 
system comprising means for moving said sensor a predetermined 
or measurable area which are much less than each of the areas 
of each sensor element. 

30 When the sensor is arranged in a flexible system will the 
apparatus quickly and accurate be able to make new measurements 
and output values for each predetermined or measured position. 

The apparatus can also comprise a computer provided with a 
35 software program for establishing a representation of the high 
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resolution image by calculating the representation of the 
received incident light from the scene at each subarea. 

The light sensor can preferably be moved by means of an 
5 electro-mechanical device or a piezo-element, so that the 
position of the sensor-matrix can be controlled electrically, 
and thus the position syncronized to the storing and 
calculating electronic devices. 

10 A further advantage of the invention, is that each set of 
output values from the sensor matrix can be transmitted or 
stored externally together with information about the exact 
position of the sensor array at the time of the scanning, for 
calculation in another place or time, for example after 

15 transmission through network or storing on digital madia. This 
will give the advantage that no extra computer resources for 
calculation has to be included in the invented apparatus, but 
existing resources in the receiving devices can be used. 

20 The invention will be explained in greater detail below, 
describing only simple idealised exemplary embodiments with 
reference to the drawing, in which 

Fig. 1 shows a sensor cell according to the invention with a 
25 matrix 16 subareas a in a first and second position. 

Fig. 2a-2c is illustrating the method according to the 
invention in a matrix with one cell-array, 

30 Fig. 3a-3c is illustrating the method according to the 
invention in a matrix with one cell-array but were there is a 
physical distance between the cells in the sensor, 

Fig. 4 is illustrating the method according to the invention in 
35 a matrix with one cell-array, but where the influence of an 
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error is minimised by means of a filter according to the 
invention. 

Fig. 5 shows schematically an apparatus according to the 
5 invention. 

Fig. 6a-6b shows schematically one embodiment of the invention, 

Fig. 7a-7b shows schematically another embodiment of the 
10 invention, 

Fig, 8a -8b shows schematically a third embodiment of the 
invention, 

15 Fig. 9a-9b shows schematically a fourth embodiment of the 
invention , and 

fig. lOa-lOc shows a fifth embodiment of the invention in form 
of a scanner. 

20 

In fig. 1 is shown a sensor cell 1 with 16 equally sized 
subareas a moved from a first scanning position 2 to a second 
scanning position 3. 

25 as can be seen there is T = 6 overlapping subareas, which means 
that the total area N of the overlapping subareas is N=6*a 
while the entire sensor area A = 16*a. 

That means that if the sensor is moved in such a way that the 
30 overlapping area only is increasing by the size a of a subarea 
during each scanning the resolution can as maximum be extended 
by the size A, 

In fig. 2a is illustrated how the original picture 4, 
35 consisting of two black squares 5 and two white squares 6, are 
being processed by the method according to the invention. The 
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black squares are given the value 0 while the white squares are 
given the value 256, which are the normal value range used in 
conputer and sensor systems. 

5 The sensor is having a cell-array 7 with 6 cells, and in the 
next line is seen how the sensor according to the invention is 
evaluating the image. Each cell is larger than each square in 
the image. The sensor will therefore not he able to see the 
correct image resolution. Since each cell only is able to see a 

10 number of squares, will each cell give a mean value from the 
squares. In this case is the first set of squares 5a and 6a and 
the second set of squares 5b and 6b each given the respective 
cells mean value of 128 (in the figs, indicated by single-lined 
right inclined hatching) . Therefore, image 8 would be the 

15 resulting image in case the image is not further processed. 

In fig. 2b is shown how the sensor is making a first scanning 
9a, then moving the sensor in the X-direction with a step a, 
and thereafter making a second scanning 9b, 
20 * 

The incoming output signals from the sensor cells is stored in 
the electronics attached to the sensor (directly or indirectly) 
as respectively a first and second set of signals based on 
each cells incoming light. 

25 

The first set of output signal from the sensor is measured as 
respectively V 0 <1,1) 4 V.(l,2), V 0 <1,3), V D (1.4), V.(1,S), V„{1,6), 
and the second set of output signals are measured as 
respectively V.<2,1). V.(2.2), V B <2,3), V.<2,4), V 0 (2,5), V.(2,6). 
30 The o stands for output, s for sensor, the numbers denotes the 
scan number and the cell number respectively. 

The new set of calculated I. light values are denoted 
respectively 1.(1,1), 1.(2,1), 1.(1,2), 1.(2,2), 1.(1,3), 1.(2,3), 
35 1.(1,4), 1.(2,4). 1.(1,5) 1.(2,5), 1.(1,6). The numbers in the 
brackets denotes the a value (-1 or 2) and the cell nuxriber in 
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the original sensor matrix that it originates from, 
respectively. 

The new picture, which is shown in fig. 2c is now calculated by 
5 means of the formula (3), as the subareas are equally sized. 

I,(z) = V # *N - II, (n) for n=l to n=N, a*z 

where 

10 

N is the number of subareas that each cell is splitted into, 
n is the index for subareas in the sensor cell, 
a is the area of a subarea of the sensor cell, 
V. is the output signals from the sensor cells , 
15 I, is the computed light radiation received by the subarea a, 
z is the subarea to be used for calculation, 
A is the area of the sensor cell. 

The resolution enhancement factor is in the present case 2, and 
20 for each cell is there a overlapping area T of 1. 

1.(1,1) = V 0 ll,l)*2 - XI. (a), in this specific case is Xl.(a) not 
known, as no values has been previously calculated, why an 
estimate is given, e.g. the mean value of the starting cell, 
25 that is 2l.(a) = VJ1. 1) . therefore: 
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1.(1,6) - V a (l,6)*2 - 1.(2,5) » 256*2-256=256 

In fig. 3 is shown how the method according to the invention is 
operation in case each sensor cell is not abutting on the next 
5 sensor cell. 

As can be seen in fig. 3a is the image the same image as in 
fig. 2a, however the sensor cells 7 are seeing the image 
differently as there is a space b between the cells. Therefore, 
10 image 10 would be the resulting image in case the image is not 
further processed. 

In fig. 3b is shown how the space can be covered by three 
scannings, because the distance between two cells is exactly 
15 half the width of a cell. The resulting resolution enhancement 
will in this case not be equal to N (which is two) as the 
scannings cover an area which where not covered by the sensor 
matrix, thereby giving a resolution enhancement of 3* 

20 The sensor is making a first scanning 10a, then moving the 
sensor in the X-direction with a step a, and thereafter making 
a second scanning 10b, moving the sensor in the X-direction 
with step a and making a third scanning 10c. 

25 The incoming input signals is stored in the sensor electronics 
attached to the sensor directly or indirectly as respectively a 
first, a second and a third set of signals based on each cells 
incoming light. 

30 The first set of output signal from the sensor is measured as 
respectively V 0 ( 1,1), V 0 (l,2), V.<1,3), V Q (1,4), V Q <1,5), V.U.6), 
and the second set of output signals are measured as 
respectively V, (2 ,1), V.(2,2), V e <2,3), v w <2,4), V % <2,S), ^(2.6) 
and the third set of output signals are measured as 

35 respectively V„(3,l), V 0 <3,2), V c <3,3), VJ3,4), VJ3,5), V 0 <3,6>. 
The o stands for output, the numbers denotes the scan number 
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and the cell number respectively. A third number would have to 
be introduced in case of a sensor matrix with more than one 
row. 



5 The new set of calculated I. light values are denoted 
respectively 1.(1,1) , 1.(2,1). 1,(3,1), 1.(1,2), 1.(2.2), 1.(3,2), 
1.(1,3), 1,(2,3), 1.(3,3). 1.(1,4), 1,(2,4). 1,(3.4). 1.(1,5), 
1.(2,5). 1.(3,5), 1.(1,6), The numbers in the brackets denotes 
the n value (=1, 2 or 3) and the cell number in the original 
10 sensor matrix that it originates from, respectively. A third 
number would have to be introduced in case of a sensor matrix 
with more than one row. 



The new picture, which is shown in fig. 3c is now calculated by 
15 means of the formula (3) as the subareas of the sensor cell are 
equally sized. 

X.(z) = V.*N - £l.(n) for n=l to n=N, a*z 
20 where, 

N is the number of subareas that each cell is splitted into, 
n is the index for subareas in the sensor cell, 
a is the area of a subarea of the sensor cell, , 
25 V G is the output signals from the sensor cells, 

I. is the computed light radiation received by the subarea a, 
z is the subarea to be used for calculation, 
A is the area of the sensor cell. 



30 1.(1,1) - V.(l,l}*2 - V.(l.l) 

1.(2.1) - V„(2,l)*2 - 1.(1,1) 

1.(3,1) = V 0 (3,l)*2 - 1.(2.1) 

1.(1,2) « V.(l,2)*2 - 1.(3,1) 



« 256*2-2S6-256 (White) 
= 256*2-256=256 (White) 
- 256*2-256=256 (White) 
= 256*2-256-256 (White) 



35 Same values until 1.(1,3): 
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la both cases is made an estimate of the initial value of 
1,(1,1). In case this estimate is wrong or contains an error 
will the mistake propagate throughout the calculations. 

15 In fig. 4 is shown how one embodiment of the digital filter 
according to the invention is used for correcting an error. 

The original picture 11 in (4) has a grey background (in the 
figs, indicated by single-lined right inclined hatching) with 

20 value 128 and two grey dots (12a, 12b) with value 64 (in the 
figs . indicated by small cross-hatching) . With one scan 13 the 
resulting picture would be 14. According to the invention two 
scans 15a, 15b are used in this example to double the 
resolution, A=2. The sensor cells measure the grey background 

25 correctly with values 128, but the output V.(2,2) (output from 
scan 2, sensor cell 2) is due to a distortion or other error, 
measured as 80 instead of 96 , which would be the correct value. 
This will cause all the calculated new values after 1.(1,2) to 
be incorrect 16, which can be seen by the following 

30 calculation: 

1.(1,1)= V.(l,l)*2- V 0 {1,1)= 128*2-128 =128 (Estimated initial 
value) 

1.(2,1)= V.(2.1)*2- 1.(1,1)= 128*2-128 =128 
35 1.(1,2)= V.(l,2)*2- 1.(2,1)= 128*2-128 =128 
1.(2,2)= V.(2,2)*2- 1.(1,2)= 80*2-128 =32 
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1.(1,3)= V.(l,3)*2- 1.(2,2) = 96*2-32 =160 
1.(2,3) = V.(2,3)*2- 1.(1,3) = 96*2-160 -32 
1.(1,4)= V c (1.4)*2- 1.(2,3)= 96*2-32 =160 
1.(2,4)= V 0 (2,4)*2- 1.(1,4)= 128*2-160 =96 
5 1.(1,5)= V 0 (1.5>*2- 1.(2,4)* 96*2-32 =160 
1.(2,5)= V.(2,5)»2- 1.(1,5)= 128*2-160 =96 
1.(1,6)= V 0 {1,6)*2- 1.(2,5)* 96*2-32 =160 

It is obvious that the error introduced in V 0 (2,2) will 
10 influence on all the following calculations, and will cause an 
oscillation even when V. has stopped changing. This is shown in 
the resulting picture 16a. 

In the calculations of the I. values in 16b a very simple 
15 digital filter according to (4) has been introduced. This 
digital filter has the advantage that it can use more values 
from previous measured and/or calculated cells to calculate 
new I. values, thereby minimising the influence of noise and 
other errors: 

20 

The filter coefficients are chosen to: 
F v =2 

F(-l,0)=0.75 

25 F(-2,0)=0.25 (only one row in the sensor matrix) 
All other coefficients in (4) are set to 0. 

1.(1,1) * V,(l,l)*2- V Q (1,1) 
128*2-128 

30 « 128 (Estimated initial value) 

1.(2,1) = V # {2.1)*2-0. 75*1.(1,1) -0.25*1.(1,1) 
2*128-0.75*128+0.25*128 
128 

(1.(1,1) are used two times, as no present value exist) 
35 1.(1.2) = V o (l,2)*2-0. 75*1.(2,1) -0.25*1.(1,1) 

2*128-0.75*128+0.25*128 
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« 128 

1.(2,2) = V o (2,2)*2-0. 75*1.(1, 2) -0.25*1.(2,1) 
2*80-0 .75*128+0 .25*128 
32 

5 1.(1,3) * V.(l, 3) *2-0. 75*1.(2. 2) -0.25*1.(1,2) 

2*96-0.75*32+0.25*128 
136 

1.(2,3) * V o (2.3)*2-0.75*I.<1.3)-0.25*l.(2,2) 

2*96-0.75*136*0.25*32 
10 = 82 

1.(1,4) » V o (l,4>*2-0. 75*1.(2, 3) -0.25*1.(1,3) 

2*96-0.75*82+0.25*136 

96.5 

1.(2,4) « V 0 (2, 4) *2-0. 75*1.(1, 4) -0.25*1.(2, 3) 
15 = 2*128-0.75*96.5+0.25*82 

163 

1.(1.5) » V.(l, S)*2-0. 75*1.(2, 4)-0. 25*1.(1. 4) 
2*128-0. 75*163+0. 25«96. 5 
109.5 

20 1.(2.5) = V o (2,5)*2-0. 75*1.(1, 5)-0. 25*1.(2, 4) 
» 2*128-0.75*109.5+0.25*163 
133 

1.(1,6) « V.(1,6)*2-0.75*I.(2,5)-0.25*I.(1,5) 
2 *128-0 .75*109 . 5+0 . 25*163 
25 « 129 

In fig. 4 the different values in accordance with the 
calculation above are indicated by means of different types of 
hatching • 



30 



As can be seen from this example, the error introduced in the 
measurement of cell V.(2,2) will not cause oscillations in the 
following calculations and the influence will disappear. The 
resulting picture can be seen in 16b. 



35 
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Pig. 5 illustrated in form of an apparatus 17 by way of example 
the basic principle for using the invention in a web camera. 

The apparatus 17 comprises a light sensor 18 with a number of 
5 cells 19 formed as a matrix. The sensor serves for receiving 
incident light radiated from a scene (not shown) . The light 
sensor is movable mounted on a frame 20. Two drivers 21 are 
furthermore arranged for cyclically moving the light sensor 
along the frame in a X and Y direction. 

10 

The drivers could e.g. be a piezo-element or an electro- 
mechanical device. 

Each driver is associated with a position sensor 22 for 
15 recording the immediate position of the light sensor relative 
to the frame and for generating output representing said 
position. 

A computer 23 serves for receiving the generated output from 
20 the position sensors and for. by means of a software program, 
calculating control values which via a circuit 24 is send to 
the two position drivers 21 for bringing these to drive the 
light sensor 18 in such a way that the steps of movement as 
function of the scannings defines a predetermined curve in a 
25 coordinate system. 

The apparatus functions in principle in the way described above 
with reference to the figures 1-16. 

30 More specific , the light sensor 18 is by means of an activator 
(not shown) cyclically brought to scan the scene (not shown) a 
number of times. 

Simultaneously with the scanning the light sensor is stepwise 
35 moved an identical number of times along the frame in the x-Y 
direction while defining a number of subareas, whereby the 
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total distance covered during said movements of the light 
sensor corresponds to the extend of the cell and the distance 
to the next cell in the X-Y direction, respectively. 

5 During said operation of the apparatus output representing the 
received incident light from the scene at each sensor cell is 
send from the light sensor 18 to the computer 23 via another 
circuit 25, which is shown in bold line. 

10 By storing the received output from the light sensor and adding 
information about the position of the sensor at the time of 
scanning by means of another software program, the computer can 
transmit the pictures and added information to another computer 
(not shown) by means of the circuit 26 which can be any kind of 

15 wired or wireless connection or network. The receiving 
computer at the other end of the network can then establish a 
representation of the high resolution picture by calculating 
the representation of the received incident light from the 
scene at each subarea. 

20 

The calculated representation of the high resolution picture 
can then be shown on different kind of displaying devices as 
computer screens and projectors - 

25 The above mentioned curve which is showing the steps of 
movement as function of the scannings in a coordinate system 
can within the scope of the invention in itself have any 
suitable form. In the following advantages and drawbacks of 
some representative curves are mentioned. 

30 

In fig. 6a and 6b is illustrated a simple principle for moving 
a light sensor 26, which in this case is only moved in the Y- 
direction. Xt is, however, obvious that the movement as well 
could be in the x-direction. 

35 
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Fig 6b shows a coordinate system where the time of scanning is 
shown on the axis of abscissas and the position of the light 
sensor on the axis of ordinate. 

5 The scannings are performed with equidistant time intervals and 
the distance between each position of the light sensor 26 is 
identical to these intervals. Thereby the movement of the light 
sensor as function of the scannings needs to precisely follow a 
triangle curve as shown in fig. 6b. 

10 

Fig. 7a illustrates a sensor matrix 27 moved in the diagonal 
direction, which is simple to control merely mechanical since 
the movements only take place in one direction. 

15 The subsequent calculations are, on the other hand, relatively 
complicated as the subareas are having different sizes, which 
need to be converted by means of interpolation to., equal 
quadratic pixels elements before being able to show the high 
resolution picture on e.g. a television screen. 

20 

As shown in fig. 7b are the scanning positions asynchronous 
with scanning times. That means that each new scanning is taken 
place in a matrix position not scanned before whereby an almost 
infinite high resolution is possible by only scanning a 
25 sufficient number of times. 

The distances between the positions of the sensor matrix and 
the scanning times, respectively, are as in the example shown 
in fig 6a and 6b equidistant thereby requiring that the curve 
30 shown in fig. 7b must be a triangle. 

In fig. 8a the sensor matrix 28 is carrying out free 
oscillations which is the most simple way for mechanically 
moving the sensor. The oscillations shall, however, still be 
35 controlled in such a way that they are kept within the wanted 
limitations. 
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Fig. 8b shows a situation where the distance between the 
scannings is equal in time whereby the distances between the 
positions of the sensor matrix 28 will be different resulting 
5 in complicated subsequent calculations and reduced precision of 
this calculations . 

Fig, 9a and 9b show the same as in fig. 8a and 8b. The only 
difference is that the distances between the positions of the 
10 sensor matrix now are constant whereas the distance in time 
between the scannings is varying. This mode of using the 
invention requires in many cases a picture buffer as most 
receivers, such as computers or servers or the like will 
require that the pictures arrives with equal distances in time. 

15 

Fig* 10a, 10b and 10c illustrate the invention used in a 
scanner. 

in a scanner a sensor matrix having three arrays usually is 
20 used. Each array having a red (R) , green (G) or blue (B) 
filter. All colours are scanned in one scan (single pass) while 
a step motor is moving the three arrays over the object to be 
scanned . 

25 Some scanners are using only one array. In this case will the 
filter be changed between each scan, i. e. that the array shall 
pass the object three times, (multipass) . 

The resolution in a scanner is depending on the number of 
30 elements in the array and of the possible precision of the 
movement of the arrays. Typically will the resolution in the 
movement direction of the array be the double of the resolution 
in the direction of the array itself, conventionally 300x600 
dpi or 600x1200 dpi. The distances between each pixel cause 
35 this, which typically is in the same order of magnitude as the 
pixel itself, whereby a higher resolution is possible in the Y- 
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direction where the step motors can move in steps of precisely 
one pixel . 

By using the invention in a scanner the size of the pixel will 
5 not any more be the restricting factor as the array can be 
moved in the X-direction by means of an arrangement similar to 
the arrangement shown in fig. 6a. 

The movement in the Y-direction can be carried out by means of 
10 a step motor, but in even smaller steps independent of the size 
of the pixels whereby the wanted resolution can be obtained. 

This arrangement is shown in fig. 10 showing a scanner 29 
having three arrays 30, 31 and 32 of cells R (red) , G (green) 
15 and B (blue) . 

One driver 33 are arranged for cyclically moving the arrays in 
the X-direetion. 

20 The driver is associated with a position sensor 34 for 
recording the immediate position of the light sensor in the X- 
direction and for generating output representing said position. 

A computer 35 serves for receiving the generated output from 
25 the position sensors and for, by means of a software program, 
calculating control values which via a circuit 36 is send to 
the position driver 33 for bringing this to drive the three 
arrays cyclically in a wanted mode along the X-axis as 
illustrated in fig. 10b, while the three arrays is moved along 
30 the Y-axes by means of a stepmotor as illustrated in fig. 10c. 



25 



11 ✓09X07 16:53 HOLME PATENT (VS NR. 915 t?26 

23 Modtaget PVS 

* f SEF. 2001 

Claims 

1. A method for converting a low resolution first image 
produced by means of a light sensor (18; 19) for receiving 
5 incident light radiated from a scene to a high resolution 

second image, whereby said light sensor is constructed as an 
array (30;31;32) or a matrix (27?28) having a number of 
cells (1;19), each defining a predetermined area, comprising 
- that said first image is cyclically scanned a number 
10 of times, 

that the light sensor simultaneously with the scanning 
stepwise is moved an identical number of times relative 
to the light radiating scene in at least one direction 
thereby defining a number of subareas, 
15 - that the total distance covered during movement of the 

light sensor corresponds to the extend of the cell or to 
to the extend of the cell plus a distance defined by or 
to the neighbour cell in said at least lone direction, and 
that a representation of the high-resolution second image 
20 is established by calculating the representation of the 

received incident light from the scene at each subarea. 

2. A method according to claim 1, comprising that the light 
sensor (18; 19} for each step is moved a distance 

25 corresponding to the extend of the area covered by the cell 

in the direction of movement or to the extend of the area 
covered by the cell in the direction of movement plus a 
distance defined by or to the neighbour cell in the 
direction of movement divided with the number of scanning 

30 times* 

3. A method according to claim l r con^rising that the light 
sensor (18; 19) for each step is moved a distance 
corresponding to the extend of the area covered by the cell 

35 plus a distance defined by or to the neighbour cell minus 

the extend of the area of the smallest subarea to be 
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calculated in the direction of movement, divided with the 
number of scanning times. 

4* A method according to claim 1, comprising that the light 
5 sensor (18; 19) is moved asynchronous to the scannings. 

5. A method according to each of the claims 1 - 4, comprising 
that the high resolution second image is computed by 
calculating the value for each subarea by means of the 

10 formula: 

I,(z) = V 0 *A/a(z) - Zljn) for n=l to n-N, n*z 
where 

15 

N b the number of subareas that each cell is splitted into, 
n = the index for subareas in the sensor cell, 
a a the area of a subarea of the sensor cell, 
V D = the output signals from the sensor cells, 
20 i a = the computed light radiation received by the subarea a, 

z = the subarea to be used for calculation, 
A = the area of the sensor cell » 

6. A method according to each of the claims 1-5, comprising 
25 - stepwise moving the light sensor by means of at least one 

driver relative to the light radiating scene, 
- recording the immediate position of the light sensor by 
means of at least one position sensor generating output 
representing said sensor position, 
30 - sending said output to a computer having a software 

program for calculating control values on basis of the 
received output, and 

sending signals representing the calculated control 
values to the at least one position driver for bringing 
35 this to drive the light sensor in such a way that the 
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steps of movement as function of the scannings defines a 
predetermined curve. 

7. a method according to claim 6, comprising that the 
5 predetermined curve describing the steps of movement as 

function of the scannings in a coordinate system is formed 
as triangle or a stairway or is sinusoidal. 

8. A method according to each of the claims 1-7, comprising 
10 that the light sensor (18; 19) is moved by means of a 

piezoelectric-element or an electro-mechanical device, 

9. A method according to each of the claims 1-7, comprising 
that the scene is moved relative to the light sensor (18; 19) 

15 by means of a piezo-element or an electro-mechanical device. 

10. A method according to each of the claims 1-9, comprising 
that the light sensor (18; 19) is moved in a system of x-y 
coordinates and moved at least in one of the x-y directions 

20 of this system or in the direction of a cell of the light 

sensor. 

11. A method according to each of the claims 1 - 10, 
comprising that the method further comprises a step for 

25 minimising the influence of errors in previous calculated or 
estimated values. 

12. A method according to each of the claims 1 - 11, 
comprising that the influence of errors in previous 

30 calculated or estimated values is minimised by means of a 

digital filter. 

13. A method according to claim 12, comprising that the 
digital filter is using several values from several cycles 

35 to compute a filter output value by the following formula 
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IJnX A ,<nYJ = (T ut ) *V c <p UJ X f q m Y) - (F,, ? ) *V 0 <p a> X, q rl) Y) - (F M ) * 
- <F A J *V 0 (p lAJ X,q {Jk> Y> - (G^.,) *I.<n. l X fc ,m.,YJ - (G..^) 

*I l (nA f m. l Y 4 )-(Q. l( . l )*I.{nA^A) 
-<0. B .^)*I.(n a X A ,m. ft Y,). 

5 

where, 



I 4 (nX A ,mYJ is the light radiation value of the n,m'th 
element in the new X^Y^ matrix to be calculated, 
10 - I.(n_^x 4 , m^YJ is the previous calculated element of the 

n-l,m'th element of the new X A ,Y A matrix, 
<G-i..o> is the belonging predetermined filter value, 
I.<n^X A ,m. l Y A ) is the previous calculated element of the 
n,m-l'th element of the new X.,Y A matrix, 
15 - (G_„ _ x ) is the belonging predetermined filter value, 

^.t^A'^-pX*) is the previous calculated element of the n- 
n,m-m'th element of the new X^,Y A matrix, 
CG-n.-J is the belonging predetermined filter value, 
* v »<P(ii x *Q(iiY) is the measured and stored output value of 
20 the p, q'th element in the physical sensor matrix 

overlapping the n, m'th element in the new calculated 
Xa,Y A matrix from the first position, 
(F t x ) is the belonging predetermined filter value, 
- p,q are calculated from n/A and m/A, 
25 - V|(p w X,q w Y) is the measured and stored output value of 

the p,q'th element in the physical sensor matrix 
overlapping the n, m'th element in the new calculated 
X A ,Y A matrix from the A'th position, 

< p *.*> is the belonging predetermined filter value, and 
30 p,q are calculated from n/A and m/A. 



14. An apparatus (17) for converting a low resolution first 
image to a high resolution second image, conjprising 
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a light sensor (18) for receiving incident light 
radiated from a scene to a high resolution second image, 
whereby said light sensor is constructed as an array 
<30;31;32) or a matrix (27;28) having a number of cells 
5 (1;19), each defining a predetermined area, 

means for bringing the apparatus cyclically to scan the 
first image a number of times by means of the light 
sensor. 

- means (21; 33) for simultaneously with the scanning 
10 stepwise to move the light sensor an identical number of 

times relative to the light radiating scene in at least 
one direction, whereby the total distance covered during 
said movement of the light sensor corresponds to the 
extend of the cell or to the extend of the cell plus a 
15 distance defined by or to the neighbour cell in said at 

least one direction and a number of subareas are defined, 
and 

means (23; 35) for establishing a representation of the 
high resolution second image by calculating the 
20 representation of the received incident light from the 

scene at each subarea . 

15. An apparatus for converting a low resolution first image 
to a high resolution second image, comprising, 

25 - a light sensor (1;19) for receiving incident light 

radiated from a scene to a high resolution second image, 
whereby said light sensor is constructed as an array 
(30; 31; 32) or a matrix (27; 28) having a number of cells, 
each defining a predetermined area, 

30 a frame for movable mounting said light sensor, 

an activator for bringing the apparatus cyclically to 
scan the low resolution first image a number of times by 
means of said light sensor, 

at least one driver (21;33) for simultaneously with the 
35 scanning stepwise to move the light sensor an identical 

number of times relative to the light radiating scene in 
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at least one direction, whereby the total distance 
covered during said movement of the light sensor 
corresponds to the extend of the cell or to the extend of 
the cell plus a distance defined by or to the neighbour 
5 cell in said at least one direction and a number of 

subareas are defined, and 
- at least one position sensor for recording the immediate 
position of the light sensor relative to the frame and 
sending output representing said position to a computer 

10 having a software program for calculating control values 

on basis of the received output and sending signals 
representing the calculated control values to the at 
least one position driver for bringing this to drive the 
light sensor in such a way that the steps of movement as 

15 function of the scannings defines a predetermined curve 

in a coordinate system. 

16. An apparatus according to claim 14 or 15, comprising that 
a representation of the high resolution second image is 
20 established by calculating the representation of the 
received incident light from the scene at each subarea by 
means of the formula; 



25 



I„{z) » V c *A/a<z) - £l.(n) for n=l to n=N, n#z 
where 



N - the number of subareas that each cell is split ted into, 
n - the index for subareas in the light sensor cell, 
30 a ■ the area of a subarea of the light sensor cell. 

V 0 = the output signals from the light sensor cells, 

I. = the computed light radiation received by the subarea a, 

z - the subarea to be used for calculation, 

A - the area of the light sensor cell. 

35 
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17. An apparatus according to claim 14 or 15, comprising that 
the apparatus further comprises a digital filter arranged 
for minimising the influence of at least one error in the 
value . 

5 

18. An apparatus according to claim 14 or 15, comprising that 
the digital filter are using several values from several 
cycles to compute a filter output value by the following 
formula 

10 

I. <nX A ,mY A ) = (F M > *V e (p (U X, q m Y> - <P ltJ ) *V, (p (a ,X, q (1> Y ) - (F,.,) * 

" <*V>> *V,(P u> X,q w Y) - (G^.,) n f (nA,m,YJ - (G.,^) 

*I A <n e X>,in t YJ - (G_ 1# _J *l.{n.,X t> ,m>YJ 
-«5. w .^)*l.(n.A.m,Y A ) # 

15 

where , 

I A (nX»,mYJ is the light radiation value of the n,m'th 
element in the new X A ,Y A matrix to be calculated, 

20 - iJn.^m^Yj is the previous calculated element of the n- 
l,m'th element of the new X A# Y A matrix, 
(G. lt . 0 ) is the belonging predetermined filter value, 
I.(n_ 0 X 1 ,m_ 1 YJ is the previous calculated element of the 
n,m-l'th element of the new X A ,Y A matrix, 

25 - (G- e ,-i> is the belonging predetermined filter value, 

I .( n - x i l .ro HI YJ is the previous calculated element of the n- 
n, m-m'th element of the new X^Y^ matrix, (G. a ^) is the 
belonging predetermined filter value, 

v o<Pci> x »Qri> Y ) is th« measured and stored output value of 
30 the p,q'th element in the physical sensor matrix 

overlapping the n.m'th element in the new calculated X A ,Y A 
matrix from the first position, 

<F I#l ) is the belonging predetermined filter value, 
p,q are calculated from n/A and m/A, 
35 - V 0 (p u> X,q U) Y) is the measured and stored output value of 
the p,q'th element in the physical sensor matrix 
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overlapping the n, m' th element in the new calculated X A , 
Y A matrix from the A'th position, 

(F Ah ) is the belonging predetermined filter value, and p.q 
are calculated from n/A and m/A. 

5 

19. An apparatus according to claim 14 or 15, comprising that 
the computer (23; 35) is provided with a software program for 
establishing a representation of the high resolution image 
by calculating the representation of the received incident 

10 light from the scene at each subarea. 

20. An apparatus according to claim 14 or 15, coirprising 
thatthe predetermined curve is formed as a triangle or a 
stairway or is sinusoidal, 

IS 

21. An apparatus according to claim 14 or 15, conjprising that 
the light sensor (1;19) is moved in system of x-y co- 
ordinates and moved at least in one of the x-y directions. 

20 22. An apparatus according to each of the claim 14 or 15 , 
comprising that the at least one driver (21; 33) is an 
electro-mechanical device or a piezoelectric-element. 

23. A use of the method according to claims 1-13 and the 
25 apparatus according to claims 14 - 21 in a web camera or a 

still picture camera or camcorder or a monitoring camera or 
a 360° camera or a digital binocular glasses or a scanner. 
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Method and apparatus for producing a high resolution image 

ABSTRACT 

5 A method and apparatus serves for converting a low resolution 
first image to a high resolution second image. The apparatus is 
comprising a light sensor <1;19) for receiving incident light 
radiated from a scene. The light sensor has a number of cells, 
each defining a predetermined area, and is arranged for 

10 cyclically scanning the low resolution first image a number of 
times while at least one driver (21; 33) is moving the light 
sensor an identical number of times in at least one direction. 
For each step the light sensor is moved a distance 
corresponding to the extend of the area covered by the cell in 

15 the direction of movement while the total distance covered 
corresponds to the extend of the cell in said direction. 
Thereby a number of subareas are defined. A computer (23; 35) 
serves for establishing a representation of the high resolution 
second image by calculating the representation of the received 

20 incident light from the scene at each subarea by means of a 
soft ware program. Thereby is obtained a higher resolution and 
a better image quality than hitherto known. 

25 

Fig. S 



11/09/07 16:56 HOLME PATENT ft/S 

31 



34 



11/09/07 16:57 HOLME PATENT fVS 



NR. 915 035 




35 



11/09/07 16:57 HOLME PATENT P/S 



NR. 915 036 



2/10 

6a 60 5b 6 ^ 



Mod tage t P Vs 
1 ; «fc 




Fig. 2a 




Fig. 2b 




1,(2.1) i.(i^) ixw) ui.3) ww) m,4) yi.4) m^) ^,5) ui,6) 

Fig. 2c 



36 



11/09/07 IS: 57 HOLTE POTENT fVS 




_* 



NR. 915 (737 

Modtaget PVS 
' J ShP. 2001 



3/10 



s 




37 



11/09/07 



16:57 



HOU-E PATENT fVS 



NR. 915 P38 




38 



11/09/07 16:58 HOLME PATENT A/S 



NR. 915 P39 




39 



11/09/87 16:58 HOLME PATENT fl/S m - 915 P40 



Mocftaget PVS 
' 1 SEP. 200/ 



6/10 




40 



11/09/07 16: SB HOU*H PATENT fVS 



W3.915 D41 




41 



11/-09/07 16:59 HOLME PATENT A/S NR. 915 



Mocffagef pvs 
' ' SEP. 2001 

8/10 




42 




43 



11/09/07 



16:59 



HOLME PATENT A/S 



NR. 915 



044 




10/10 

3 



CM 

CO- 



CO" 



O- 







: CJ 


1 CO 


ct 


' (£ 


) CD 


et 


C5 


, CD 


at 


CD 


CD 


a 


(5 


CO 


at 


CD 


CO 


at 


CD 


CD 


Ct 


CD 


CD 


at 




CO 


at 


(D 


CO 


Ot 


CD 


CO 


at 


CD 


CO 


at 


0 


CO 


ai 


CD 


CD 


at 


(5 


GO 


at 


CD 


CO 


Ot 


CD 


CD 


ct 


CD 


CO 


at 


CD 


CO 


cs 


CD 


CO 


ess 


CD 


CO 


Ot 


CD 


CO 




15 







CD 


cdN 


at 


CD 


CO 




CD 


CD / 




O 








CO 








a 


CD 


CO 




CD 


CO 


Dt 


CD 


CO 


i . at 


CD 


co 


as 


CD 


CD 


at 


0 


CD 


az 


CD 


CD 


at 


CD 


CO 



S3 





1:31 


urn 


ill 


mm 


ill 


\im 


1131 





::c::c: 



EES 



8 



8 



Modtaget PVS 
I 1 SEP. 2001 



O 



44 



